<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
	<title>Keyword marker token filter | ElasticSearch 7.7 权威指南中文版</title>
	<meta name="keywords" content="ElasticSearch 权威指南中文版, elasticsearch 7, es7, 实时数据分析，实时数据检索" />
    <meta name="description" content="ElasticSearch 权威指南中文版, elasticsearch 7, es7, 实时数据分析，实时数据检索" />
    <!-- Give IE8 a fighting chance -->
    <!--[if lt IE 9]>
    <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
    <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
    <![endif]-->
	<link rel="stylesheet" type="text/css" href="../static/styles.css" />
	<script>
	var _link = 'analysis-keyword-marker-tokenfilter.html';
    </script>
</head>
<body>
<div class="main-container">
    <section id="content">
        <div class="content-wrapper">
            <section id="guide" lang="zh_cn">
                <div class="container">
                    <div class="row">
                        <div class="col-xs-12 col-sm-8 col-md-8 guide-section">
                            <div style="color:gray; word-break: break-all; font-size:12px;">原英文版地址: <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.7/analysis-keyword-marker-tokenfilter.html" rel="nofollow" target="_blank">https://www.elastic.co/guide/en/elasticsearch/reference/7.7/analysis-keyword-marker-tokenfilter.html</a>, 原文档版权归 www.elastic.co 所有<br/>本地英文版地址: <a href="../en/analysis-keyword-marker-tokenfilter.html" rel="nofollow" target="_blank">../en/analysis-keyword-marker-tokenfilter.html</a></div>
                        <!-- start body -->
                  <div class="page_header">
<strong>重要</strong>: 此版本不会发布额外的bug修复或文档更新。最新信息请参考 <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html" rel="nofollow">当前版本文档</a>。
</div>
<div id="content">
<div class="breadcrumbs">
<span class="breadcrumb-link"><a href="index.html">Elasticsearch Guide [7.7]</a></span>
»
<span class="breadcrumb-link"><a href="analysis.html">Text analysis</a></span>
»
<span class="breadcrumb-link"><a href="analysis-tokenfilters.html">Token filter reference</a></span>
»
<span class="breadcrumb-node">Keyword marker token filter</span>
</div>
<div class="navheader">
<span class="prev">
<a href="analysis-keep-words-tokenfilter.html">« Keep words token filter</a>
</span>
<span class="next">
<a href="analysis-keyword-repeat-tokenfilter.html">Keyword repeat token filter »</a>
</span>
</div>
<div class="section">
<div class="titlepage"><div><div>
<h2 class="title">
<a id="analysis-keyword-marker-tokenfilter"></a>Keyword marker token filter<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/keyword-marker-tokenfilter.asciidoc">edit</a>
</h2>
</div></div></div>

<p>Marks specified tokens as keywords, which are not stemmed.</p>
<p>The <code class="literal">keyword_marker</code> filter assigns specified tokens a <code class="literal">keyword</code> attribute of
<code class="literal">true</code>. Stemmer token filters, such as
<a class="xref" href="analysis-stemmer-tokenfilter.html" title="Stemmer token filter"><code class="literal">stemmer</code></a> or
<a class="xref" href="analysis-porterstem-tokenfilter.html" title="Porter stem token filter"><code class="literal">porter_stem</code></a>, skip tokens with a <code class="literal">keyword</code>
attribute of <code class="literal">true</code>.</p>
<div class="important admon">
<div class="icon"></div>
<div class="admon_content">
<p>To work properly, the <code class="literal">keyword_marker</code> filter must be listed before any stemmer
token filters in the <a class="xref" href="analysis-custom-analyzer.html" title="Create a custom analyzer">analyzer configuration</a>.</p>
</div>
</div>
<p>The <code class="literal">keyword_marker</code> filter uses Lucene’s
<a href="https://lucene.apache.org/core/8_5_1/analyzers-common/org/apache/lucene/analysis/miscellaneous/KeywordMarkerFilter.html" class="ulink" target="_top">KeywordMarkerFilter</a>.</p>
<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-keyword-marker-tokenfilter-analyze-ex"></a>Example<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/keyword-marker-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<p>To see how the <code class="literal">keyword_marker</code> filter works, you first need to produce a token
stream containing stemmed tokens.</p>
<p>The following <a class="xref" href="indices-analyze.html" title="Analyze API">analyze API</a> request uses the
<a class="xref" href="analysis-stemmer-tokenfilter.html" title="Stemmer token filter"><code class="literal">stemmer</code></a> filter to create stemmed tokens for
<code class="literal">fox running and jumping</code>.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">GET /_analyze
{
  "tokenizer": "whitespace",
  "filter": [ "stemmer" ],
  "text": "fox running and jumping"
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/940.console"></div>
<p>The request produces the following tokens. Note that <code class="literal">running</code> was stemmed to
<code class="literal">run</code> and <code class="literal">jumping</code> was stemmed to <code class="literal">jump</code>.</p>
<div class="pre_wrapper lang-text">
<pre class="programlisting prettyprint lang-text">[ fox, run, and, jump ]</pre>
</div>
<p>To prevent <code class="literal">jumping</code> from being stemmed, add the <code class="literal">keyword_marker</code> filter before
the <code class="literal">stemmer</code> filter in the previous analyze API request. Specify <code class="literal">jumping</code> in
the <code class="literal">keywords</code> parameter of the <code class="literal">keyword_marker</code> filter.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">GET /_analyze
{
  "tokenizer": "whitespace",
  "filter": [
    {
      "type": "keyword_marker",
      "keywords": [ "jumping" ]
    },
    "stemmer"
  ],
  "text": "fox running and jumping"
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/941.console"></div>
<p>The request produces the following tokens. <code class="literal">running</code> is still stemmed to <code class="literal">run</code>,
but <code class="literal">jumping</code> is not stemmed.</p>
<div class="pre_wrapper lang-text">
<pre class="programlisting prettyprint lang-text">[ fox, run, and, jumping ]</pre>
</div>
<p>To see the <code class="literal">keyword</code> attribute for these tokens, add the following arguments to
the analyze API request:</p>
<div class="ulist itemizedlist">
<ul class="itemizedlist">
<li class="listitem">
<code class="literal">explain</code>: <code class="literal">true</code>
</li>
<li class="listitem">
<code class="literal">attributes</code>: <code class="literal">keyword</code>
</li>
</ul>
</div>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">GET /_analyze
{
  "tokenizer": "whitespace",
  "filter": [
    {
      "type": "keyword_marker",
      "keywords": [ "jumping" ]
    },
    "stemmer"
  ],
  "text": "fox running and jumping",
  "explain": true,
  "attributes": "keyword"
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/942.console"></div>
<p>The API returns the following response. Note the <code class="literal">jumping</code> token has a
<code class="literal">keyword</code> attribute of <code class="literal">true</code>.</p>
<div class="pre_wrapper lang-console-result">
<pre class="programlisting prettyprint lang-console-result">{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "whitespace",
      "tokens": [
        {
          "token": "fox",
          "start_offset": 0,
          "end_offset": 3,
          "type": "word",
          "position": 0
        },
        {
          "token": "running",
          "start_offset": 4,
          "end_offset": 11,
          "type": "word",
          "position": 1
        },
        {
          "token": "and",
          "start_offset": 12,
          "end_offset": 15,
          "type": "word",
          "position": 2
        },
        {
          "token": "jumping",
          "start_offset": 16,
          "end_offset": 23,
          "type": "word",
          "position": 3
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "__anonymous__keyword_marker",
        "tokens": [
          {
            "token": "fox",
            "start_offset": 0,
            "end_offset": 3,
            "type": "word",
            "position": 0,
            "keyword": false
          },
          {
            "token": "running",
            "start_offset": 4,
            "end_offset": 11,
            "type": "word",
            "position": 1,
            "keyword": false
          },
          {
            "token": "and",
            "start_offset": 12,
            "end_offset": 15,
            "type": "word",
            "position": 2,
            "keyword": false
          },
          {
            "token": "jumping",
            "start_offset": 16,
            "end_offset": 23,
            "type": "word",
            "position": 3,
            "keyword": true
          }
        ]
      },
      {
        "name": "stemmer",
        "tokens": [
          {
            "token": "fox",
            "start_offset": 0,
            "end_offset": 3,
            "type": "word",
            "position": 0,
            "keyword": false
          },
          {
            "token": "run",
            "start_offset": 4,
            "end_offset": 11,
            "type": "word",
            "position": 1,
            "keyword": false
          },
          {
            "token": "and",
            "start_offset": 12,
            "end_offset": 15,
            "type": "word",
            "position": 2,
            "keyword": false
          },
          {
            "token": "jumping",
            "start_offset": 16,
            "end_offset": 23,
            "type": "word",
            "position": 3,
            "keyword": true
          }
        ]
      }
    ]
  }
}</pre>
</div>
</div>

<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-keyword-marker-tokenfilter-configure-parms"></a>Configurable parameters<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/keyword-marker-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<div class="variablelist">
<dl class="variablelist">
<dt>
<span class="term">
<code class="literal">ignore_case</code>
</span>
</dt>
<dd>
(Optional, boolean)
If <code class="literal">true</code>, matching for the <code class="literal">keywords</code> and <code class="literal">keywords_path</code> parameters ignores
letter case. Defaults to <code class="literal">false</code>.
</dd>
<dt>
<span class="term">
<code class="literal">keywords</code>
</span>
</dt>
<dd>
<p>
(Required*, array of strings)
Array of keywords. Tokens that match these keywords are not stemmed.
</p>
<p>This parameter, <code class="literal">keywords_path</code>, or <code class="literal">keywords_pattern</code> must be specified.
You cannot specify this parameter and <code class="literal">keywords_pattern</code>.</p>
</dd>
<dt>
<span class="term">
<code class="literal">keywords_path</code>
</span>
</dt>
<dd>
<p>(Required*, string)
Path to a file that contains a list of keywords. Tokens that match these
keywords are not stemmed.</p>
<p>This path must be absolute or relative to the <code class="literal">config</code> location, and the file
must be UTF-8 encoded. Each word in the file must be separated by a line break.</p>
<p>This parameter, <code class="literal">keywords</code>, or <code class="literal">keywords_pattern</code> must be specified.
You cannot specify this parameter and <code class="literal">keywords_pattern</code>.</p>
</dd>
<dt>
<span class="term">
<code class="literal">keywords_pattern</code>
</span>
</dt>
<dd>
<p>(Required*, string)
<a href="http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html" class="ulink" target="_top">Java
regular expression</a> used to match tokens. Tokens that match this expression are
marked as keywords and not stemmed.</p>
<p>This parameter, <code class="literal">keywords</code>, or <code class="literal">keywords_path</code> must be specified. You
cannot specify this parameter and <code class="literal">keywords</code> or <code class="literal">keywords_pattern</code>.</p>
<div class="warning admon">
<div class="icon"></div>
<div class="admon_content">
<p>Poorly written regular expressions can cause Elasticsearch to run slowly or result
in stack overflow errors, causing the running node to suddenly exit.</p>
</div>
</div>
</dd>
</dl>
</div>
</div>

<div class="section">
<div class="titlepage"><div><div>
<h3 class="title">
<a id="analysis-keyword-marker-tokenfilter-customize"></a>Customize and add to an analyzer<a class="edit_me edit_me_private" rel="nofollow" title="Editing on GitHub is available to Elastic" href="https://github.com/elastic/elasticsearch/edit/7.7/docs/reference/analysis/tokenfilters/keyword-marker-tokenfilter.asciidoc">edit</a>
</h3>
</div></div></div>
<p>To customize the <code class="literal">keyword_marker</code> filter, duplicate it to create the basis for a
new custom token filter. You can modify the filter using its configurable
parameters.</p>
<p>For example, the following <a class="xref" href="indices-create-index.html" title="Create index API">create index API</a> request
uses a custom <code class="literal">keyword_marker</code> filter and the <code class="literal">porter_stem</code>
filter to configure a new <a class="xref" href="analysis-custom-analyzer.html" title="Create a custom analyzer">custom analyzer</a>.</p>
<p>The custom <code class="literal">keyword_marker</code> filter marks tokens specified in the
<code class="literal">analysis/example_word_list.txt</code> file as keywords. The <code class="literal">porter_stem</code> filter does
not stem these tokens.</p>
<div class="pre_wrapper lang-console">
<pre class="programlisting prettyprint lang-console">PUT /my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_custom_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "my_custom_keyword_marker_filter",
            "porter_stem"
          ]
        }
      },
      "filter": {
        "my_custom_keyword_marker_filter": {
          "type": "keyword_marker",
          "keywords_path": "analysis/example_word_list.txt"
        }
      }
    }
  }
}</pre>
</div>
<div class="console_widget" data-snippet="snippets/943.console"></div>
</div>

</div>
<div class="navfooter">
<span class="prev">
<a href="analysis-keep-words-tokenfilter.html">« Keep words token filter</a>
</span>
<span class="next">
<a href="analysis-keyword-repeat-tokenfilter.html">Keyword repeat token filter »</a>
</span>
</div>
</div>

                  <!-- end body -->
                        </div>
                        <div class="col-xs-12 col-sm-4 col-md-4" id="right_col">
                        
                        </div>
                    </div>
                </div>
            </section>
        </div>
    </section>
</div>
<script src="../static/cn.js"></script>
</body>
</html>